Colors expressed in a standardized form are preferable because they can be used, for example, by a robot to recognize objects on the basis of color. Non-standardized colors are so variable that a robot can not make reliable decisions on the basis of color.
Colors expressed in standardized form are also useful for reproducing the colors in images more accurately. For example, the colors in a color image captured by a camera under illumination conditions that differ in color temperature from those for which the camera is color balanced will generally look "wrong". For example, if a camera balanced for indoor incandescent light is used in outdoor daylight conditions, colors in the outdoor-captured image will look too blue since there is more blue in the daylight than there is in the indoor incandescent light.
Digital color images can be created either by direct digital imaging on a charge-couple device ("CCD") chip, by digitization of a video camera's analog output, or by digitization of images printed on paper, transparencies or film. In each case the basic principle whereby the resulting digital pixel values composing the image colors are created is the same. In particular, the spectral power distribution (spectrum) of the light reflected from a point on a matte surface is generally the product of the spectral power distribution of the illumination incident at that point and the percent surface spectral reflectance function of the surface. For a surface with a shiny, specular component the reflected light also includes a second component which has the same spectrum as the incident illumination. In either case, the spectrum of the reflected light as a function of position forms the light entering the camera and is known as the color signal. For each pixel, the digital color image's value depends on the imaging device's spectral sensitivity functions and on the spectrum of the incoming color signal.
For sensitivity functions which are sensitive over a relatively narrow range of wavelengths and which do not overlap significantly, the change in the resulting pixel value created by a change in the incident image illumination can be approximated well by scaling by the amount by which the color of the incident illumination changed. In this context, "the color of the incident illumination" means the camera's response to an ideal white surface viewed under that illumination. If the sensitivity functions are not relatively narrow band, then a technique called "spectral sharpening" can be used to combine the output responses of the functions, prior to scaling, in order to optimize the scaling performance (see: Finlayson et. al. "Spectral Sharpening: Sensor Transformations for Improved Color Constancy", J. Opt. Soc. America, May, 1994, pp. 1553-1563). If the color of the ambient illumination can be determined, then the difference between the color of the ambient illumination and the desired illumination can be used to produce an image which is colored as if it were taken under the desired illumination. This process is often described as color correction. The present invention facilitates accurate estimation of ambient illumination and thereby facilitates image color correction.
The "grey-world" method is one well known prior art technique for estimating the color of the incident illumination. With the grey-world method, the color of the incident illumination is estimated as the average of color in the image. This method is very unreliable because the average is very unstable. For example, the image of a large field of grass will be primarily green, so the average color in the image will be green, even though the skylight illumination illuminating the grass is far from green. Color correction of such an image based on the grey-world technique produces readily perceptible color errors throughout the image.
Another prior art method of estimating the color of the incident illumination is the Retinex method (see: "The Retinex Theory of Color Vision", E.H. Land, Scientific American, 1977, pp. 108-129). This method effectively uses the maximum value found within the image for each of the three RGB color channels as the estimate of the color of the illumination. This method is unstable because it depends on the assumption that somewhere in every scene there will be a surface which is maximally reflective in each of the three color channels. This assumption is frequently violated.
Another prior art method of estimating the color of the incident illumination is described by Wandell et al in U.S. Pat. No. 4,648,051. For 3-band color images, Wandell et al calculate the best two-dimensional subspace of the three-dimensional space of image colors and then extract a model of the illumination based on the normal to that subspace. Unfortunately, this method is unreliable because the assumption that colors will lie in a two-dimensional subspace is generally violated.
Another prior art technique estimates the spectrum of the incident illumination, from which its color is easily derived, by placing a set of surfaces of known percent spectral reflectance in the image where their color will be recorded in the image. This method has limited practical utility, since in general it is not possible to include the known surfaces in the image prior to imaging the image.
The 2D convex hull gamut mapping algorithm is another prior art technique, which considers the set of possible illuminants that could map the observed gamut of image pixels to a canonical gamut of expected possible pixels under the standard, known illuminant. See: "Color Constancy in Diagonal Chromaticity Space", Finlayson, Proc. IEEE Fifth Intl. Conf. on Computer Vision, June, 1995, pp. 218-223. Although gamut mapping sometimes yields more accurate results than other prior art techniques, it is more time consuming.
Von Kries adaptation is another prior-art color correction technique which can be used once the illumination is known. This is a process of scaling the RGB channels by a correction factor. See: "Necessary and Sufficient Conditions for Von Kries Chromatic Adaptation to Give Color Constancy", West et al, J. Math. Biology, 1982, pp. 249-258.
U.S. Pat. No. 5,351,079 Usui describes a method of estimating the illumination by using a 3-input, 3-output neural network as a decorrelator to minimize the correlation between the R, B and G bands of a color image. The neural network by itself does not accomplish color constancy; instead, it is trained to decorrelate the R, G, and B bands from one another. Usui's neural network uses only 6 weights, depends on unsupervised instead of supervised learning, uses a feedback instead of a feed forward process, and functions only to decorrelate the signals in the color bands. Moreover, Usui performs passive input-image-independent color correction. In other words, Usui's correction steps are fixed once his neural network has been trained. By contrast, the present invention uses the input image data to adaptively determine the color correction to be applied.